HttpWebRequest hits 1s timeout when resolving http://localhost on Win10 (.NET Framework is fast) #23581

jimevans · 2017-09-17T15:29:43Z

Consider the following method (error/exception handling omitted for brevity:

public virtual string MakeHttpRequest(Uri fullUri, string httpMethod, string requestBody)
{
    string responseString = string.Empty;
    HttpWebRequest request = HttpWebRequest.Create(fullUri) as HttpWebRequest;
    request.Method = httpMethod;
    request.Timeout = 30000;
    request.Accept = "application/json,image/png";
    if (request.Method == "POST")
    {
        string payload = requestBody;
        byte[] data = Encoding.UTF8.GetBytes(payload);
        request.ContentType = "application/json;charset=utf-8";
        Stream requestStream = request.GetRequestStream();
        requestStream.Write(data, 0, data.Length);
        requestStream.Close();
    }

    HttpWebResponse webResponse = request.GetResponse() as HttpWebResponse;
    using (Stream responseStream = webResponse.GetResponseStream())
    using (StreamReader responseStreamReader = new StreamReader(responseStream, Encoding.UTF8))
    {
        responseString = responseStreamReader.ReadToEnd();
    }
    return responseString;
}

Executing this method against a server running on localhost yields wildly different execution times when run via the full .NET Framework vs. .NET Core. I have a demo project that shows the discrepancy, making calls to a locally running HTTP server, that server being the Chromium project's chromedriver.exe, used to automate the Chrome browser. When running the executables created by that project, you can see the following typical outputs (as run on Windows 10, build 16288):

Run against .NET Core 2.0:

Starting test...
Starting ChromeDriver 2.32.498550 (9dec58e66c31bcc53a9ce3c7226f0c1c5810906a) on port 9515
Only local connections are allowed.
Started session 946394531499259b7442073c0e26060d
Navigating to http://www.google.com
Navigation complete
Making 10 HTTP calls to localhost, logging the elapsed time...
Elapsed time for HTTP call: 1038 milliseconds
Elapsed time for HTTP call: 1016 milliseconds
Elapsed time for HTTP call: 1015 milliseconds
Elapsed time for HTTP call: 1011 milliseconds
Elapsed time for HTTP call: 1012 milliseconds
Elapsed time for HTTP call: 1024 milliseconds
Elapsed time for HTTP call: 1013 milliseconds
Elapsed time for HTTP call: 1027 milliseconds
Elapsed time for HTTP call: 1013 milliseconds
Elapsed time for HTTP call: 1014 milliseconds
Test finished. Press to exit.

Running against the full .NET Framework 4.5:

Starting test...
Starting ChromeDriver 2.32.498550 (9dec58e66c31bcc53a9ce3c7226f0c1c5810906a) on port 9515
Only local connections are allowed.
Started session 81c2bbd21a0d89354b2dead8d52ee982
Navigating to http://www.google.com
Navigation complete
Making 10 HTTP calls to localhost, logging the elapsed time...
Elapsed time for HTTP call: 35 milliseconds
Elapsed time for HTTP call: 7 milliseconds
Elapsed time for HTTP call: 12 milliseconds
Elapsed time for HTTP call: 5 milliseconds
Elapsed time for HTTP call: 4 milliseconds
Elapsed time for HTTP call: 7 milliseconds
Elapsed time for HTTP call: 4 milliseconds
Elapsed time for HTTP call: 6 milliseconds
Elapsed time for HTTP call: 5 milliseconds
Elapsed time for HTTP call: 5 milliseconds
Test finished. Press to exit.

I would expect similar performance between framework versions. Absent that, I would expect there to be settings to modify to the HttpWebRequest object that would yield similar performance as a workaround.

[EDIT] C# syntax highlighing by @karelz

The text was updated successfully, but these errors were encountered:

jimevans · 2017-09-17T15:36:47Z

I'll point out here, to forestall the inevitable comment mentioning it, that changing from System.Net.HttpWebRequest to System.Net.Http.HttpClient in the .NET Core case doesn't resolve the issue. The performance is the same. In the aforementioned demo project, there is an alternative implementation that can be used in the .NET Core case that utilizes the newer class. The performance is comparable.

davidsh · 2017-09-17T16:02:34Z

This is a duplicate of #23255.

jimevans · 2017-09-17T17:47:53Z

@davidsh While I'm more than happy to concede that it might be (and probably is) a duplicate, the statistics given in the report of #23255 seem to indicate individual requests via the full framework take longer than the same requests using .NET Core. This is the opposite behavior of what I've observed, hence this new issue report. I'm sure I'm missing some context or am misinterpreting what I'm reading in that report. Please feel free to educate me.

qmfrederik · 2017-09-17T20:32:34Z

Can you try changing your tests so that they use http://127.0.0.1 in the URL instead of http://localhost?

When using 127.0.0.1, I get response times of ~10ms, whereas localhost give me response times of ~1020ms, so it looks like there's a 1-second timeout you're hitting.

Drawaes · 2017-09-17T23:35:21Z

Which might suggest it's an issue with the dns lookup class rather than the http class. Which would be insightful

jimevans · 2017-09-18T11:33:34Z

Of course, @qmfrederik is correct. Using the IP address directly in the URL reduces the time required to make the HTTP call. However, as pointed out by @Drawaes, that does indicate an issue somewhere else in the network stack, most likely in DNS name resolution, and maybe specific to local loopback addresses. If someone with the appropriate permissions wants to edit the title of the issue report to better indicate what the issue actually is, that would be fine by me. Additionally, if this is still a duplicate of a different issue, and someone could point me to the proper one, I'd like that too.

Drawaes · 2017-09-18T11:36:39Z

Try using the dns class directly to resolve local host.

Drawaes · 2017-09-18T11:38:22Z

Also if you can try your test against ::1 It might be resolving localhost to ipv6 and that might be the issue.

jimevans · 2017-09-18T13:32:34Z

@Drawaes The server in question only listens on IPv4 in dual-stack environments. Attempting to change the URL to ::1 (predictably) yields a System.Net.WebException with the message, "An error occurred while sending the request. A connection with the server could not be established." Dns.GetHostEntry() returns both an IPv6 and an IPv4 address for the localhost host name in my environment. This leads me to three questions:

How do I force the HttpWebRequest (or the HttpClient, for that matter) to use the IPv4 address in that case?

How is it that the full framework version, which returns the same list of addresses from Dns.GetHostEntry(), and in the same order, is able to bypass the issue?

Is it not a valid issue that there is a negative perf difference between .NET Core and the .NET Framework for the same code?

Drawaes · 2017-09-18T13:50:35Z

@jimevans I don't work for MS so can't speak for them I am merely an interested party but let me try to answer you :)

I don't know other than to resolve the DNS entry yourself and call the IPv4 address (I have had to do this before on full framework, not for perf but other reasons).
That is a good question, it should at least be consistent that I agree with!
Absolutely, I hope I didn't come across as saying your issue wasn't valid. I was just trying to narrow it down for everyone (binary search of the problem space ;) ).

qmfrederik · 2017-09-18T17:58:45Z

What I think happens is that .NET resolves localhost to ::1 and 127.0.0.1. Because ::1 is first in the list, it tries to connect to ::1 (perhaps with a 1-second timeout), fails, and continues with the next entry, 127.0.0.1, which succeeds.

Looking at it this way, it seems the framework is behaving correctly. You could argue that using localhost to connect to an IPv4-only service on a dual-stack system is not optimal, and that you should use 127.0.0.1 instead.

As to why the difference, it seems Desktop .NET gives preference to IPv4 whereas .NET Core treats them equally. @davidsh or @stephentoub may be able to comment as to whether that is by design.

Out of curiosity, is there any reason using 127.0.0.1 instead of localhost would not work for your use case?

jimevans · 2017-09-18T18:38:23Z

@qmfrederik Your supposition of what's happening under the covers is what I suspect too, but given that the IP addresses appear in the same order in both Desktop and Core configurations, I would expect the underlying library to approach them the same way.

As to your question, the API that my project provides to end users includes the user providing a URL to a remote-end processor (like the chromedriver.exe mentioned in the repro), and this URL is often to the local machine. Knowing the level of sophistication of my average user, insisting on the loopback IP address means I have no elegant choices. I can tell all of my users to use an IP address instead of localhost. This would put the .NET language bindings at odds with every other language binding providing this API, as neither Java, Python, Ruby, nor JavaScript have this restriction. Or, I could put my language bindings in the business of parsing the URL I get, and magically attempting to do the right thing, which is going to be error-prone, and isn't a core competency of the project.

Moreover, I have no control over the remote-end behavior with respect to IPv4 vs. IPv6. It's entirely possible that the several remote ends that my project supports, which are provided by browser vendors (including Microsoft!) may differ between what support they offer for different network stacks.

bofcarbon1 · 2017-10-09T01:12:28Z

I've started using .NET Core 2.0 with Entity Framework Core 2.0. A lot of changes from .NET 4.5 for me as I used the Http Response Message. Now I'm returning an 'object' that automatically gets converted to Json. I've received some response handshake errors that worry me. Thinking I'm getting timed out sometimes. I am sending requests via an observable from an Angular 2 node.js web app. My web page is loading without the results from the .NET api. Eventually it will load but sometimes it looks like it takes 25 or more seconds. Have not measured but it seems like a lifetime in web response time. I use some console.log commands and notice that they show up late. Granted I'm still learning the observable and am not certain about any implied REST asynchronous thing going on. As a user though I'm not going to want to see a page without data that suddenly appears later. I'd just start to reload the page after a few seconds. I use Google Chrome usually but I gave IE a try just to compare and its about the same. I am also using Visual Studio 2017 Community (free) run on IIS Express (localhost) and my Angular web app runs on the Cloud 9 IDE (free version) so my issue could be limited resources. I've also noticed that Visual Studio 2017 Community IDE is slow to respond. Just switching files takes a while. It often freezes and goes unresponsive. I'm leaning toward Visual Studio as the weak link. My Angular 2 web apps with an observable usually get a quick response back when calling Express against MongoDB. If I continue to have problems with Visual Studio I'm thinking that I may look to Java Spring REST as an alternative to Express. Java is still a big market and I don't want to stare at my laptop waiting for my back end tools to deliver services.

vincywindy · 2017-10-09T07:25:51Z

I use this simple code:

    class Program
    {
        static string url = "http://192.0.0.39:6277/check";
        static void Main(string[] args)
        {

            Do();
            Console.ReadLine();
        }
        public static async void Do()
        {
            while (true)
            {
                var g = Guid.NewGuid().ToString();
                Console.WriteLine(DateTime.Now.ToString()+ " 发送:" + g);
                var result= await HttpPostMath(url, g);
                Console.WriteLine(DateTime.Now.ToString()+" 接受:" + g);
                await Task.Delay(1000);
            }
        }
        public async static Task<string> HttpPostMath(string url, string paramsValue)
        {
            try
            {
                using (var client = new HttpClient())
                {
                    var content = new StringContent(paramsValue);
                    var response = await client.PostAsync(url, content);
                    var responseString = await response.Content.ReadAsStringAsync();
                    return responseString;
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.ToString());
                return paramsValue;
            }
        }
    }

The server return the string that what I send.
Here is the result:
In Win10(I7 4770+16GBmen), it is no problem:

But in Win2008(E5-2650+16GBmen) it is slow:

This code accesses a LAN server without any complex operations.
I try another two Win10 and win server 2008 machine,Same result。
So I think something was wrong in Win7/2008 when use Http,Win10 doesn't have this problem.
By the way,when I use .net framework 4.6, 2008 and win10 both work well.

[EDIT] Fix code formatting by @karelz

pieter-venter · 2017-12-09T03:06:20Z

We were hit by this very same issue. We use Selenium extensively and noticed that on Windows 10 machines our tests slowed to a crawl when we switched to .Net core. What is strange is on virtual machines, running Windows 2016 they were still running really fast.

I found @jimevans's demo project and started experimenting.

I tried disabling IPv6 on the network adapter but it made no difference. It seems like "localhost" uses some special loopback adapter so disabling IPv6 on your other network adapters won't do anything!

This solved the problem for me:
https://superuser.com/questions/586144/disable-ipv6-loopback-on-windows-7-64-bit/681967#681967

I also confirmed that Windows 2016 servers already have this ::ffff:0:0/96 higher on the priority list, hence why they don't suffer from the 1000ms delay.

@jimevans It may be an option to make the ServerUrl in DriverService.cs read/write so that we can set it to 127.0.0.1 as a workaround.

karelz · 2017-12-29T03:03:35Z

Updated title
To sum up the problem: When using http://localhost, .NET Core can hit 1s timeout on Win10 resolving it to ::1 first, then to 127.0.0.1.

Workaround: Resolve IP yourself, don't use http://localhost.

Open question: Why is .NET Core different than .NET Framework? Can it be fixed?

davidsh · 2017-12-30T05:41:37Z

@Drawaes The server in question only listens on IPv4 in dual-stack environments. Attempting to change the URL to ::1 (predictably) yields a System.Net.WebException with the message, "An error occurred while sending the request. A connection with the server could not be established." Dns.GetHostEntry() returns both an IPv6 and an IPv4 address for the localhost host name in my environment. This leads me to three questions:

This is the root cause of the problem. Why does this server only listen on IPv4 addresses, especially for loopback? That is not optimal. And disabling IPv6 is not a good idea since IPv6 is really baked into Windows.

The problem, however, is not due to DNS resolution, per se. But rather trying to determine quickly which IP address to use for a given DNS name.

When a DNS name is resolved, there might be multiple results, i.e. multiple A records. Or in many cases, both A and AAAA (IPv6) records returned.

.NET Framework HTTP stack uses managed sockets and that has optimizations for parallel connection attempts on both IPv6 and IPv4 addresses for a given DNS name. In the old days, when IPv6 was new, some clients didn't support it. But if the DNS records return both IPv6 and IPv4 addresses, then each one needs to be tried before a connection is made. If IPv6 is enabled on the client machine (or server) then, the connect will fail (timeout). Then the IPv4 connection is tried. Doing this one at a time is slow. However, recent Windows OS versions can do the TCP connection for both IPv4 and IPv6 at the same time. And the first one that connects will win usually. So, it's much faster.

.NET Core on Windows uses a different HTTP stack (native WinHTTP). I think it is not using the winsock call that does parallel TCP connection lookups. So, that is why it might be slower in this case where IPv6 connections are disabled on the loopback listener side.

There is no way that I know of to disable how loopback resolves. It is built into the Windows TCP drivers. You can't suppress getting the "::1" and "127.0.0.1" addresses for a single "localhost" DNS lookup.

My recommendation on this is to make sure you listen on all loopback addresses (both IPv4 and IPv6).

jimevans · 2017-12-30T07:23:04Z

@davidsh

My recommendation on this is to make sure you listen on all loopback addresses (both IPv4 and IPv6).

While I don’t disagree that is the ideal solution, I feel compelled to reiterate that consumers of the .NET Core API of the project I hit this issue with have no control over how the server ends listen. The server in question can come from any of a list of major browser vendors (including Google via the Chromium project, Mozilla, Apple, and even Microsoft). If I could modify the behavior of the listening component, I absolutely agree that having it listen on all loopback addresses would be the correct option, but that is not the case.

noobed7 · 2018-01-11T20:45:32Z

@davidsh

.NET Core on Windows uses a different HTTP stack (native WinHTTP). I think it is not using the winsock call that does parallel TCP connection lookups.

How do you know that? Could you point me to some code in the dotnet repository to see how things are laid out? I've tried to find any information on that, but with no luck.

karelz · 2018-01-12T02:43:44Z

@noobed7 see code in System.Net.Http and System.Net.Http.WinHttp:

https://github.com/dotnet/corefx/blob/c84ed430941fcf4e475b1149558498c55b4e295e/src/System.Net.Http/src/System.Net.Http.csproj#L184-L191

https://github.com/dotnet/corefx/tree/c84ed430941fcf4e475b1149558498c55b4e295e/src/System.Net.Http.WinHttpHandler/src/System/Net/Http

karelz · 2018-01-17T23:13:26Z

Triage: It seems there is not much we can do with current WinHTTP-based stack and there is workaround (use IP address / listen on both IPv4 and IPv6). We should be able to address the issue with ManagedHandler. Moving to Future.

Change Web Cmdlets Tests to Use 127.0.0.1 instead of localhost due to dotnet/corefx#24104 This provides a decent speed boost to the WebCmdlet tests (Faster in dozens of times).

iSazonov · 2018-02-01T04:27:14Z

@karelz Many thanks for the workaround!

https://github.com/dotnet/corefx/issues/24104

davidsh · 2018-04-17T22:17:47Z

We should be able to address the issue with ManagedHandler. Moving to Future.

@geoffkizer @stephentoub Do we expect this issue to be resolved now with SocketsHttpHandler? Is it efficient in dealing with IPv4/IPv6 resolution especially IPv6 doesn't resolve correctly?

stephentoub · 2018-04-18T13:27:58Z

Is it efficient in dealing with IPv4/IPv6 resolution especially IPv6 doesn't resolve correctly?

SocketsHttpHandler simply relies on Socket.ConnectAsync:

https://github.com/dotnet/corefx/blob/ab8e81453fa869556821c464bd5824b588d679cd/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectHelper.cs#L57-L62

I've not tested in this context to see what impact that has on this case.

geoffkizer · 2018-04-18T19:00:54Z

Me either. I'm also not sure exactly what .net framework is doing here. This seems like something worth understanding better...

* Revert "Pull PSReadLine from PSGallery" (PowerShell#5986) This reverts commit beffdcf. * update 6.0.1 change log (PowerShell#5937) * Remove Fedora 25, Add Fedora 27 (PowerShell#5984) Fedora 25 was EOL on December 12, 2017 https://fedoramagazine.org/fedora-25-end-life/ Signed-off-by: Eli Uriegas eli.uriegas@docker.com * tests: fix function to test for docker OS due to change to use linuxkit for mac (PowerShell#5843) * Also, fix syntax issues to allow to work with released Pester * Start using Travis-CI cache (PowerShell#6003) * Fix a NullReference exception in Enter-PSHostProcess (PowerShell#5995) * Return better error for 'pwsh -WindowStyle' on unsupported platforms. (PowerShell#5975) * Change Microsoft.PowerShell.Commands.SetDateCommand.SystemTime to struct. (PowerShell#6006) Change Microsoft.PowerShell.Commands.SetDateCommand.SystemTime class to struct and resolve the error in Set-Date cmdlet - SetLocalTime function is causing a parameter error (error code 0x00000057). * Add Simplified multipart/form-data Support to Web Cmdlets Through -Form Parameter (PowerShell#5972) - Adds -Form Parameter to Invoke-WebRequest and Invoke-RestMethod. - Form Accepts any IDictionary. - Keys are used as multipart/form-data field names (PSObject unwrapped and ConvertTo<String>(Object)). - FileInfo values are added as StreamContent with application/octet-stream content type and the FileInfo.Name as the file name. - Strings are treated as StringContent. - Singe values are converted to string with ConvertTo<String>(Object) and treated as StringContent - Top level collections are enumerated and converted as above. Nested collections are treated as a single value and converted accordingly. - Form is mutually exclusive with -Body and -InFile. - Per PowerShell-Committee decision, -Form makes no assumptions about the HTTP method used. It can theoretically be used with any method. User will need to manually supply -Method POST to post the form. - ContentType and content related headers supplied to -Headers will be ignored/cleared as MultipartFormDataContent requires control of these headers. * Add '-settingsfile' to 'pwsh' to support loading a custom powershell config file. (PowerShell#5920) Support loading a custom `powershell.config.json` file via the command-line for use in testing. This change supports replacing the default `powershell.config.json` file that's usually loaded from the `PSHome` directory with a custom version file. The primary use-cases for this command-line option are as follows: 1. Allow the CI system to disable settings that impact test run times; such as disabling syslog usage on Linux and MacOS 2. Support testing of syslog and os_log without interfering with normal PowerShell operations during test runs via launching an instance with custom log settings. * Fix .gitignore (PowerShell#5991) Use single '*' in file name /src/TypeCatalogGen/powershell*.inc * update date on change log (PowerShell#6023) * Skip tests that use ExecutionPolicy cmdlets on Unix (PowerShell#6021) * CI Build: Use TRAVIS_PULL_REQUEST_SHA to accurately get the commit message (PowerShell#6024) * Rework Windows Start menu folder name (PowerShell#5891) Changes the Windows start menu folder name from ProductSemanticVersionWithName to ProductSemanticVersionWithNameAndOptionalArchitecture - now the start menu is `PowerShell` for all versions. * update processes to allow for coordinated vulnerability disclosure (PowerShell#6042) * Resort parameter list in alphabetical order (PowerShell#6052) After PowerShell#5920 we should reorder parameter list (PowerShell#4989) * new `Issue-Announcement` label (PowerShell#6041) * Adds script to patch the shortcut working directory to the user's home folder. (PowerShell#6043) Update the shortcut created by the installer to use %HOMEDRIVE%%HOMEPATH% for the working directory. * Fix "Pull Request Process" dead link (PowerShell#6066) Fix "Pull Request Process" anchor links in governance.md * Add Environment Variable override of telemetry (PowerShell#6063) Inspired by dotnet cli, and allows for Telemetry opt-out on packaging systems such as AppImage and Snapcraft where the filesystem is immutable. * add comments about where most of the packages are from (PowerShell#6016) * Test: Add Verbosity and more accurate timeout implementation for Start-WebListener (PowerShell#6013) - Add verbosity to the WebListener when it fails to assist in troubleshooting - Switch the WebListener initialization timeout to count cycles instead of using fixed dates to work around possible VM CI sleep/ issues. * Build: Update PowerShell to build with .NET Core SDK 2.1.4 (PowerShell#6054) * Change Web Cmdlet Tests to Use 127.0.0.1 not Localhost (PowerShell#6069) Change Web Cmdlets Tests to Use 127.0.0.1 instead of localhost due to dotnet/corefx#24104 This provides a decent speed boost to the WebCmdlet tests (Faster in dozens of times). * Remove unsupported members from the enum 'Language' in Add-Type (PowerShell#5829) [breaking change] Remove the unsupported members (various versions of CSharp and `JScript`) from the enum 'Language' in Add-Type. After this change, `Add-Type -Language` only supports `CSharp` and `VisualBasic`. * Doc: Fix the command to build type catalog in internals.md (PowerShell#6084) * Fix the filtering of analytic events on Unix platforms. (PowerShell#6086) - Remove previous `UseAlwaysAnalytic` workaround in `SysLogProvider.Log` - Update Script Block logging to always log to the operational channel with `UseAlwaysOperational` - Fix `PSChannel` on Linux to use a bitmask - Handle `UseAlwaysOperational` and `UseAlwaysAnalytic` keywords but adding to `_keywordFilter` if the associated channels were selected in the configuration * [Feature] Remove -TimeOutSec from non timeout related tests (PowerShell#6055) The timeout mechanism should be supported by the test framework (Pester here). * change logo to SVG (PowerShell#6072) Current README uses a PNG which doesn't scale for high dpi displays (starts to look fuzzy). Created 64 pixel wide version of svg of the black logo and inserted into README.md. Note that it currently doesn't render as the absolute URL points to master so that it gets rendered correctly in other places where the README.md is used (like docker hub). * Enable setting PSSession Name when using SSHTransport and added Transport property (PowerShell#5954) When using New-PSSession -SSHTransport -Name ... the Name parameter wasn't being used when creating the remoterunspace so an automatically generated one was always created. Fix is to explicitly pass the transport name when generating a runspace name including other transports like VMBus and Container. Refactor runspace name generation code to produce "RunspaceN" and added a Transport property. Tests will have to wait until we have the infrastructure for remoting tests and Enable-SSHRemoting work is complete. Fix PowerShell#5951 Fix PowerShell#2426 * Add registry value for easy detection (PowerShell#6094) Add registry value for easy detection. For example, the presence of a single key of 6.0.* x64. This is much faster than MSI detection. Note there are currently two GUIDs, one for 6.0.* x64 and one for 6.0.* x86. fixes PowerShell#6090 * Update test framework and tests to support 4x version of Pester (PowerShell#6064) * Test fixes and changes needed to support Pester 4.0.8 * Replace 'Should Contain' with new 'Should FileContentMatch' assertion Explicitly check for string creation with write-output * Use the current version of pester and install it in modulesDir * Simplify logic for relative path test. Multiple '..' is not needed for a relative path, a single one will do. Also, on multi-drive systems using split-path -noqualifier will probably do the wrong thing with regard to constructing a correct path. Remove extraneous Should Not Throw test, if this throws, the test will fail, we don't need to explicitly assert the not throw * In some environments it is possible that computer name is 'localhost', so that should be allowed * [feature] Add link for migrating tests from Pester v3 to v4 Fix up capitalization and white space issues Change one test to check FullyQualifiedErrorId rather than just `Should Throw` * [feature] update invoke-item test to handle the case where multiple notepad processes are running * Fix spelling issue with Pester 4x, calling it Pester 4 should be sufficient * Use 'RequireAdminOnWindows' tag in Set-Date tests (PowerShell#6034) * Use 'RequireAdminOnWindows' tag instead of 'Test-IsElevated' function. * Fix incorrect condition of user privilege in Linux/macOS CI * Update 'Restore-PSPester' to use 'Save-Module' (PowerShell#6112) Update Restore-PSPester to use 'Save-Module' to get the latest version of Pester. We have moved to the latest version of Pester via PR PowerShell#6064 * Msi installer: Add smoke test (PowerShell#6105) PR 6043 broke the installer (issue PowerShell#6095). To prevent this from happening in the future, add a smoke test that installs the msi in appveyor build and make build fail if installation failed. It uses the exit code to determine the success. The reason why it does not fail in the current state is because as I pointed out here, the failing custom action is not returning its exit code. * add scripts to set/update the release tag in VSTS (PowerShell#6107) * Improve table view for Certs and Signatures by adding EnhancedKeyUsageList and StatusMessage (PowerShell#6123) * add EnhancedKeyUsageList to default table view of X509Certs add StatusMessage to default table view of cert signatures * set fixed width of 20 to Subject so more room for EnhancedKeyUsageList content change EnhancedKeyUsageList to script to just show the friendly name * Get-ChildItem <PATH>/* -file should include <Path> as search directory (PowerShell#5431) * get-childitem <PATH>/* -file should include <Path> as search directory * [Feature] Added check for -Directory and more tests * [Feature] Added check for the dynamic parameter type * Refactor the Get-Content tests to use -TestCases. (PowerShell#6082) * Update examples to 6.0.1 and removed a removed parameter in Install-PowerShellRemoting.ps1 (PowerShell#6060) The PowerShellVersion parameter no longer exists in Install-PowerShellRemoting.ps1. Updated examples to 6.0.1 from alpha 9 * add guidance on adding copyright and license header to new source files (PowerShell#6140) * Update Raspbian installation instructions to create pwsh symlink (PowerShell#6122) * Update Raspbian installation instructions to create pwsh symlink * [breaking-change] Fix range operator (PowerShell#5732) Breaking-change: "0".."9" returns [char] previously in PowerShell Core (6.0.0, 6.0.1), now it returns [int]. After the change, the behavior is the same as in Windows PowerShell. * Update copyright and license headers (PowerShell#6134) Based on standard practices, we need to have a copyright and license notice at the top of each source file. Removed existing copyrights and updated/added copyright notices for .h, .cpp, .cs, .ps1, and .psm1 files. Updated module manifests for consistency to have Author = "PowerShell" and Company = "Microsoft Corporation". Removed multiple line breaks. Separate PR coming to update contribution document for new source files: PowerShell#6140 Manually reviewed each change. Fix PowerShell#6073 * Invoke-Item.Tests.ps1 handles finding multiple ping executables. (PowerShell#6120) Invoke-Item.Tests.ps1 handles finding multiple ping executables. Fixes Issue PowerShell#5220 * Enable $env:PAGER to work correctly if arguments are used (PowerShell#6144) Correctly parse $env:PAGER with args like 'less -w'. * Support running tests in root privilege on Linux. (PowerShell#6145) Support running tests in root privilege on Linux by adding the `REQUIRESUDOONUNIX` tag. - Delete skip tag in tests which require `sudo`. * fix spelling failures in CI (PowerShell#6191) This fixes spelling failures in CI. It appears to be caused by a change in the tool used to test spelling This change: -Updates the dictionary for new and words which are detected differently -updates markdown where it is more appropriate -adds one file to the markdown tests. * Add tests for Set-Item Cmdlet for Function Provider. (PowerShell#6166) * change the registry version detection value to semver from the default value (PowerShell#6192) change the registry version detection value to semanticversion from the default value fixes PowerShell#6171 * Revert PR 6043 - Adds script to patch the shortcut working directory to the user's home folder (PowerShell#6203) Revert PR 6043 - Remove failing script from MSI reverted from commit 795c611 * [feature] Make UTF-8 Default Encoding for application/json (PowerShell#6109) When a charset is not supplied for a JSON response, the default encoding should be UTF-8 per RFC 8259. This commit changes the default charset to UTF-8 for JSON responses when a charset is not defined. * Add options to enable PSRemoting and register Windows Event Logging Manifest to MSI installer (PowerShell#5999) * Docker package test fix and updates (PowerShell#6169) * Add common aliases for all write-* commands default message parameter (PowerShell#5816) * add common write aliases * add Message alias to the MessageData parameter for Write-Information add Msg and Message alias to the Object perameter for Write-Host * Add tests for new aliases * Add -SkipHeaderValidation Support to ContentType on Web Cmdlets (PowerShell#6018) * Add SkipHeaderValidation Support to ContentType on Web Cmdlets * Move -SkipHeaderValidation Tests to Contexts * Add ContentType -SkipHeaderValidation Tests * Improve ContentType Exception * Improve error message on invalid -ContentType * Add tests for *-Item Cmdlets in Function Provider (PowerShell#6172) * [breaking change] Throw terminating error in New-TemporaryFile and make it not rely on the presence of the 'TEMP' environment variable (PowerShell#6182) - Fixes issue PowerShell#4634 by throwing a terminating error as agreed. - Makes the command not rely on the presence of the TEMP environment variable to get path to temp directory and use the .Net method Path.GetTempPath() instead. - Catch exception more specific as given by the documentation - Improve existing test. * Don't add trailing spaces to last column when using Format-Table (PowerShell#5568) * don't add trailing space to last column in table * fix out-file tests to new behavior of no padding trailing spaces * Refactor new-msipackage into packaging.psm1 (PowerShell#6208) Refactor new-msipackage and related function into packaging.psm1 * Docs: correct a linux installation typo (PowerShell#6219) * Add Password parameter to Get-PfxCertificate cmdlet (PowerShell#6113) Add Password parameter to Get-PfxCertificate cmdlet to allow automatization instead of prompting for password every time. * fix various places that still refer to old versions of pwsh (PowerShell#6179) * Fix a typo in the help content of `pwsh` in ManagedEntranceStrings.resx * Make it clearer how and what to mark as "not applicable" in PR template (PowerShell#6209) * use powershell windowsservercore docker image for release builds (PowerShell#6226) * Set pipeline thread stack size to 10MB (PowerShell#6224) * Uses TLS 1.2 on Windows during Start-PSBootstrap (PowerShell#6235) * Fixed TLS version error on Windows when building PowerShell * Use TLS 1.2 in Start-PSBootStrap Without Breaking HTTPS (PowerShell#6236) * Add RequireSudoOnUnix tag for get-help <cmdletName> tests. (PowerShell#6223) * Fix table alignment and padding. (PowerShell#6230) - The original change to remove extra padding didn't take into account alignment. Fix logic to accommodate left, center, and right alignment in the table format and also add tests. - Fix ImplicitRemoting test that validates formatting to use same instance due to formatting changes in this PR - Only use loopback to same powershell instance for formatting test as the other tests implicitly expect Windows PowerShell. * Fix date tests failing in travis CI full builds (PowerShell#6249) * Add scripts to generate unified Nuget package (PowerShell#6167) Remove the functions which generated Nuget packages for Windows. Add function New-UnifiedNugetPackage to generate nuget packages for each assembly with unix and windows runtimes. Add function New-NuSpec and New-ReferenceAssembly for creating the required items forNew-UnifiedNugetPackage. Add a sample for cross platform project with conditional compilation for Linux. Add function Publish-NugetToMyGet to publish nuget packages to powershell.myget.or * fix MSI creation errors, and capture wixpdb for later patch creation (PowerShell#6221) - add `wixpdb` output when creating `MSI` package - capture `wixpdb` in official build - clean up anything left behind from previous MSI builds before starting MSI build to prevent using dirty files. - make sure MSI creation fails if there is an error - ignore `.wixpdb` files in git - Add functionality to `Start-NativeExecution` to - only display output if there is an error - log caller information - WXS validation error fixes - Remove unused `ExitDialog` to fix ICE82 - Add KeyPath to `SetPath` to fix ICE18 - Use `HKMU` which translates to `HKLM` to runtime to fix various validation errors about creating the shortcut - Suppress Validation errors - suppress ICE61, which is about same version upgrades being allowed - suppress ICE57, caused by the shortcut not being installed per user * revert tests marked pending in PowerShell#6230 (PowerShell#6251) * Fix the terse output on Windows for unelevated test run (PowerShell#6252) * Fix the terse output on Windows for unelevate test run * Make the regex pattern more accurate * use add instead of invoke web request in nanoserver docker file (PowerShell#6255) * change the artifact count for wixpdb (PowerShell#6254) * Fix `Start-PSPester` to include or exclude 'RequireSudoOnUnix' tag smartly on Unix (PowerShell#6241) If `-sudo` is specified, make sure to include 'RequireSudoOnUnix' tag on Unix if the 'Tag' is not specified. If `-sudo` is not specified, make sure to exclude `RequireSudoOnUnix` tag on Unix if the 'ExcludeTag' is not specified. * Add 'Path' alias to '-FilePath' parameters and others for several commands (PowerShell#5817) * Restore modules from the NuGet package cache by using dotnet restore (PowerShell#6111) * Remove AppVeyor specific cmdlet from 'Start-NativeExecution' (PowerShell#6263) Refreshed the cache by running `dotnet nuget locals all --clear` before `dotnet restore`. * FixBuild: Revert the changes to get Pester from NuGet cache (PowerShell#6282) Attempt to fix the macOS build. In PowerShell#6263, I tried flushing the cache and it worked in the PR. However, after merging the PR, it turns out the master CI build failed again with the same Pester package restore error. So I'm reverting the change that got Pester from NuGet cache. * Updated install md bin path (not packaged documentation) (PowerShell#5914) * Cleanup after Powershell install for CentOS/Fedora Docker Images (PowerShell#6264) * Adding "yum clean all" to CentOS 7 Dockerfile reduces docker image size by 76MB * Adding "dnf clean all" to Fedora Dockerfile reduces docker image size by 168MB * Refactor Web Cmdlets Tests to Pester 4 Syntax (PowerShell#6257) * Pass with Update-PesterTest * Search and Replace Pass * [Feature] Move to BeTrue * Update docs with test guidelines with RequireSudoOnUnix tag. (PowerShell#6274) * Updating testing docs with RequireSudoOnUnix tag. * Enable the pending Save-Help tests in CI (PowerShell#6289) Some CI level 'Save-Help' tests were disabled in PowerShell#2806 because HelpInfo URIs for powershell modules were broken (tracked by PowerShell#2807). However, they were forgotten to be enabled when the URI issue was fixed. This PR reenables those tests. * get-childitem -LiteralPath should accept 'Include' or 'Exclude' filter (PowerShell#5462) * get-childitem -LiteralPath should accept 'Include' or 'Exclude' filter * Test for OsLocalDateTime property of Get-ComputerInfo. (PowerShell#6253) * Test for OsLocalDateTime property in ComputerInfo. * Add TLS1.2 workaround for code coverage script (PowerShell#6299) * Update CimDSCParser to fix Configuration compilation for DSC (PowerShell#6225) Update CimDSCParser to fix Configuration compilation for DSC * MSI: Cause preview builds to install Side by side with release builds (PowerShell#6301) MSI: Cause preview builds to install Side by side with release builds - change the upgrade code if there is a preview part of the version * Make sure explorer context menu can be patched (PowerShell#6302) Make sure explorer context menu can be patched - set keypath for explorer log context menu * Tests for Get-Process Cmdlet for Module and FileVersion parameters (PowerShell#6272) * Tests for Get-Process cmdlet. * Tests for Get-Process run as admin. * Skipping some Get-Process tests on Linux * Skip test for -FileVersionInfo parameter for Linux because of the bug that cause the command to hang. * Add checks for ErrorId in Get-Process tests * Change one Get-Process test status to pending for MacOs * MSI: Make sure that file components are patchable (PowerShell#6303) MSI: Make sure that file components are patchable - avoid changing names and guids of components between builds as this prevents patch generation - This required submitting the file generated by heat - add code to make sure the generated file is not out of date * Add support for Debian in installpsh-debian.sh (PowerShell#6314) The script installpsh-debian only works for Ubuntu distributions. This PR adds support for Debian as well. Related issue: PowerShell#5700 * Remove support for Ubuntu 17.04 in installpsh-debian.sh * Rename some tests because they are duplicates (PowerShell#6312) Also remove a couple of language tests which were actually duplicated Change the one loop which loops through test cases to include an iteration number to remove test name duplication * Use new Pester syntax: -Parameter for Pester tests in Modules/CimCmdlets (PowerShell#6306) * Use new Pester syntax: -Parameter for Pester tests in Modules/CimCmdlets. * make Linux packages use correct version scheme for preview releases (PowerShell#6318) Fixes PowerShell#6315 make Linux packages use correct version scheme for preview releases - Now uses <Major>.<Minor>.<Patch>~<PreviewName> instead of <Major>.<Minor>.<Patch>-<PreviewName> as the - was interpretted as an iteration of the release not a preview. * markdown test: use strict in javascript (PowerShell#6328) * Clean build during daily build to ensure MSI package is generate correctly (PowerShell#6334) Clean build during the daily build to ensure MSI package is generated correctly - Do another clean build directly before packaging to clean up files that test has added * Build: Only restore once (PowerShell#6335) * don't restore with every publish * Exclude lines about Pester executing test scripts from terse logs (PowerShell#6336) * Rename log and logerror to Write-Log [$message] [-error] (PowerShell#6333) Fix PowerShell#6332 This change renames log and logerror functions to a single Write-Log [$message] [-error] function to avoid conflicting with the log command on MacOS. * Specify the runtime when running 'dotnet restore' in 'Start-PSBuild' (PowerShell#6345) * Build: Remove two unneeded lines from 'Invoke-AppveyorFinish' (PowerShell#6344) * Make relation-link handling in web cmdlets case insensitive (PowerShell#6338) * Make sure package verification failure fails the AppVeyor build (PowerShell#6337) * Make a relative redirect URI absolute when 'Authorization' header present (PowerShell#6325) * Add negative tests for Copy-Item over remote sessions (PowerShell#6231) * Add '-Restore' when build win-arm and win-arm64 (PowerShell#6353) * Added Service Point Manager call to force Tls12. (PowerShell#6310) Calls [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 during bootstrap to download PowerShell core to avoid an error. * Use new Pester syntax: -Parameter for Pester tests in Microsoft.PowerShell.Management module. (PowerShell#6294) * Clean the intermediate artifact 'psoptions.json' from build (PowerShell#6356) * When doing daily/test build in a non-release branch use the branch name as the preview name (PowerShell#6355) * fix errors in start-psbootstrap during release builds (PowerShell#6159) set Debian front end to non-interactive during apt-get commands, to avoid error with prompting add libffi-dev, which is required for ruby/fpm let fpm update to the latest * Minor code clean-up changes (PowerShell#5737) * Add no appimage build (PowerShell#6380) * MSI: update architecture specific filenames (PowerShell#6379) * Support non-GitHub commits in the change log generation script (PowerShell#6389) * Add lambda support to -replace operator Add support for replacement lambdas when using the -replace operator. Requires minimal changes to existing code by using the following overload: Regex.Replace(string input, MatchEvaluator evaluator) when a ScriptBlock is passed in as the replacement argument. * Pass the 'Match' object to $_ for the substitute script block in '-replace' operation (PowerShell#6029) * Update CODEOWNERS * Use new Pester syntax: -Parameter for Pester tests in engine. (PowerShell#6298) * Use new Pester syntax: -Parameter for Pester tests in engine. * Build: Remov version forcing code from NuGet package generation (PowerShell#6390) * Update version numbers to 6.0.2 (PowerShell#6402) * Add script to update the version number * Update version to 6.0.2 * Update to DotNet 2.0.6 (PowerShell#6403) Update to DotNet 2.0.6 - this addresses [Microsoft Security Advisory CVE-2018-0875: Hash Collision can cause Denial of Service](PowerShell/Announcements#4) - This is a port of PowerShell@d607f20 to 6.1 * Fix 'PropertyOnlyAdapter' to allow calling base methods (PowerShell#6394) For a PropertyOnlyAdapter, the property may come from various sources, but methods, including parameterized properties, still come from DotNetAdapter. So, the binder can optimize on method calls for objects that map to a custom PropertyOnlyAdapter. * Add configuration for https://github.com/probot/stale (PowerShell#6393) * Add yml for https://github.com/probot/stale * Standard deviation implementation on Measure-Object (PowerShell#6238) Implement iterative standard deviation algorithm for Measure-Object * Add 'NullReference' checks to two code paths related to 'PseudoParameterBinder' (PowerShell#5738) 1. Add a null check in the tab completion code on the binding result returned from `PseudoParameterBinder`; 2. Add a null check in `CommandDiscovery.LookupCommandProcessor` on the `CommandInfo` object returned from the method `LookupCommandInfo`. * Correct the '%c', '%l', '%k', '%s' and '%j' formats in 'Get-Date -UFormat' (PowerShell#4805) * Add `Waiting - DotNetCore` label (PowerShell#6406) Also, organize labels in alphabetical order. * Update to Governance doc to reflect current working model (PowerShell#6323) With some changes in the PowerShell Team, need to update Governance doc to reflect current working model being used * Make Resolve-Path -Relative return useful path when $PWD and -Path is on different drive (PowerShell#5740) * commands: make rvpa -relative do not return ./absolute_path This happens on Windows when $pwd and -path is on different drive * only return relative path inside current root * fix secret and JavaScript compliance issues (PowerShell#6408) fix secret and JavaScript compliance issues - mark secrets as a test only secrets - make javascript use strict in a specific function * Change 6.0.1 references to 6.0.2 for 'linux.md' (PowerShell#6411) * Stop 'ConvertTo-Json' when 'Ctrl+c' is hit (PowerShell#6392) * Update the 'ChangeLog.md' for 6.0.2 release (PowerShell#6417) * Update 'macos.md' and 'windows.md' with the version number '6.0.2' (PowerShell#6416) * remove runas.exe from tests as we have tags to control this behavior (PowerShell#6432) remove runas.exe from tests as we have tags to control this behavior - this should reduce the likelihood of errors * migrate the mac offical binary build to VSTS mac hosted preview (PowerShell#6363) create a PowerShellPackage...ps1 that works with these VMs (vs our custom VM) create a script to create the \powershell folder with correct permissions a YML VSTS build definition * MSI: remove the version from the product name (PowerShell#6415) MSI: remove the version from the product name During patching the original MSI information is displayed. This makes installing a patch for an MSI with a specific version very confusing. * Fix the NullRefException when using '-PipelineVariable' with 'DynamicParam' block (PowerShell#6433) * Add Alpine Linux support (PowerShell#6367) sys/sysctl header is not available in musl-libc based systems and system calls are available without any header inclusion. I have ported the cmake test from CoreFX repo (https://github.com/dotnet/corefx/blob/431475b8/src/Native/Unix/configure.cmake#L560). Progress towards: PowerShell#4605 * Change `Get-FileHash` tests to use raw bytes (PowerShell#6430) * Change filehash tests to use raw bytes * Remove testablescript.ps1 from Get-FileHash tests * Use BOM-less UTF8 input for Get-FileHash * Use new Pester syntax: -Parameter for Pester tests in Language. (PowerShell#6304) * MSI: update path with proper value (PowerShell#6441) MSI: update path with proper value - Add verification that path is updated * Fix error in windows provider when the environment as an existing set of variables name that only differs by case (PowerShell#6320) - make the provider storage for the environment on windows ignore duplicates - add tests to verify existing environment get-item behavior * Throw better parsing error when statements should be put in named block (PowerShell#6434) * Make '-CI' not depend on '-PSModuleRestore' in 'Start-PSBuild' (PowerShell#6450) * Restore for official Linux arm builds (PowerShell#6455) * Restore for official Linux arm builds (PowerShell#6455) * Use 'Unregister-Event' to remove an event subscriber when removing 'PSEdit' function (PowerShell#6449) * Fix release packaging build (PowerShell#6459) The release packaging build needs to have 'dotnet' in path, fixed that by adding Find-DotNet Refactored restore logic so it can be used from both Start-PSBuild and Copy-PSGalleryModules * Fix release packaging build (PowerShell#6459) The release packaging build needs to have 'dotnet' in path, fixed that by adding Find-DotNet Refactored restore logic so it can be used from both Start-PSBuild and Copy-PSGalleryModules * Use new Pester syntax: -Parameter for Pester tests in modules: Microsoft.PowerShell.Utility and Microsoft.WSMan.Management. (PowerShell#6366) * Add '-AsArray' parameter to 'ConvertoTo-Json' command (PowerShell#6438) Add `-AsArray` parameter to `ConvertoTo-Json` command to always pack the output string in array brackets, even if the input is a single object. * Update tests in Modules/Microsoft.PowerShell.Diagnostics to use new Pester syntax. (PowerShell#6351) * Update tests in Modules/Microsoft.PowerShell.Core to use new Pester syntax. (PowerShell#6349) * Update tests in Microsoft.PowerShell.Security folder to use Pesterv4 syntax (PowerShell#6256) * Upgrade tests in test\powershell\Host folder to PesterV4 syntax (PowerShell#6250) * Update change log for 6.1.0-preview.1 (PowerShell#6480) * Update docs for v6.1.0-preview.1 release (PowerShell#6481) * Update docs * Added the missing 'stable' label * Update stale bot message (PowerShell#6462) Fix PowerShell#6446 Add "Community members are welcome to grab these works." * Add Test-Json cmdlet (NJsonSchema) (PowerShell#5229) Resolve PowerShell#4220. The cmdlet is based on NJsonSchema. It allows to check: JSON by only parsing JSON against Schema implicitly check Schema by parsing (based on previous line check) NJsonSchema is under MIT (approved see PowerShell#5229 (comment)) * Add -Resume Feature to Web Cmdlets (PowerShell#6447) Fixes PowerShell#5964 Adds -Resume switch to Invoke-WebRequest and Invoke-RestMethod -Resume requires -OutFile Enables the ability to resume downloading a partially or incompletely downloaded file. File Size is the only indicator of local and remote file parity. If the local file is smaller than the remote file and the remote endpoint supports resume, the local file will be appended with the remaining bytes. If the local file is larger than the remote file, the local file will be overwritten If the remote server does not support resume, the local file will be overwritten If the local file is the same size as the remote file, the remote endpoint will return a 416 status code. This response is special-cased as a success in this instance. The local file remains untouched and it is assumed the file was already successfully downloaded previously. If the local file does not exist it will be created and the entire remote file will be requested. Added tests for all code new code paths (I'm pretty sure anyway) Added /Resume Controller to WebListener Documented /Resume Controller Updated .spelling to reflect terms in WebListener docs Note: I had to change the way GetResponse() tracks the current URI as we now have 3 places where the call is taking place. I don't foresee this causing any regressions. This area needs some refactoring. especially if we want to implement a retry mechanism * MSI: add function to generate a MSP (PowerShell#6445) add a function to generate an MSP * Add the parameter '-Not' to 'Where-Object' (PowerShell#6464) * Doc: Update Ubuntu source creation commands to use 'curl -o' (PowerShell#6510) Updated the Ubuntu source creation commands to remove the need for piping * Fix error in windows provider when the environment has accidental duplicates that differ only by case (PowerShell#6489) Fix error in windows provider when the environment has accidental duplicates that differ only by case. Make the provider storage for the environment on windows ignore duplicates  and only report the effective value. Add tests to verify existing environment get-item behavior and to ensure that Get-Item env:<var> reports the same as $env:<var>, namely the effective value. Fixes PowerShell#6305 and supersedes PowerShell#6320 based on discussion in PowerShell#6460. * Update Dockerfile test to use ubuntu 17.10 as the base image (PowerShell#6503) * Make sure that the width of the header is at least the size of the label (or propertyname) (PowerShell#6487) * Enable [Environment]::OSVersion to return current OS rather than compatible version (PowerShell#6457) * add application manifest dictating compatibility so that osversion shows appropriate os version on Windows * Change the 'SaveError' method in Parser to use `nameof` for error ids (PowerShell#6498) Many error messages in PowerShell currently use a LINQ expression to pass both the name of the error and the message through in the error processing. This PR uses the `nameof` feature to carry the error name/ID and gets rid of LINQ expression reflection, hopefully improving performance, especially in editor scenarios. * Use new Pester syntax: -Parameter for Pester in SDK and Provider tests (PowerShell#6490) * Add PowerShell logging tests for macOS and Linux (PowerShell#6025) This PR fixes the logging issue on Linux where logging is initialized before `-settingsFile` is parsed causing custom log settings to be ignored. (see ConsoleHost.cs and ManagedEntrance.cs) The PR also includes basic logging tests for Linux and MacOS. PSSyslog.psm1 contains the functions to retrieve selected logged items (based on PowerShell's log id and a timestamp) and Logging.Tests.ps1 contains tests for Linux and MacOS. * Make LanguagePrimitive.GetEnumerable treat 'DataTable' as Enumerable (PowerShell#6511) * Make LanguagePrimitive.GetEnumerable treat 'DataTable' as Enumerable * Use new Pester syntax: -Parameter for Pester tests in modules: PowerShellGet, PackageManagement, PSReadLine (PowerShell#6488) * Fixed scenarios where `NullOrEmpty` string was used instead of `-BeNullOrEmpty` (PowerShell#6535) * Remove empty `Should -Be` statements (PowerShell#6536) * Simplify the paths the MSI uses (PowerShell#6442) - Implementation of PowerShell/PowerShell-RFC#115 (If anything changes in the RFC, we will treat it as a bug, and fix it later) - Update registry and directory paths to use 6 for the version for stable and 6-preview for a preview release - Add checkbox to set path - default checkbox to off for preview builds and on for stable builds * Remove duplicate 'Restore-PSPackage' (PowerShell#6544) * Mark Save-Help PackageManagement tests as pending (PowerShell#6545) * Update 'Update-Help' to save help content in user scope by default (PowerShell#6352) Add the parameter `-Scope` to `Update-Help`, which takes `AllUsers` or `CurrentUser`. The default value is `CurrentUser`. * Clean up workflow logic in the module loading component (PowerShell#6523) Clean up workflow logic in the module loading component. Workflow module is not supported in PSCore. Currently, Import-Module throws a terminating error when seeing .xaml modules. After the change, Import-Module throws a non-terminating error when seeing .xaml modules. * Updating solution file with new project type guids * Removing invalid resources to make Visual Studio builds compile. * Add Unix project to sln * Map configurations to correct configurations This reverts commit 9cbf601. * Fix formatting of tables where headers span multiple rows (PowerShell#6504) * In cases where the header spans multiple rows, need to correctly calculate whitespace and trim appropriately * Use System.Span<int> and C# 7.2 language in SMA * Added new ref assemblies to Files.wxs refactor tests to remove similar xml content added single column test case * add comment about project guids * change project type for unix project * add mappings for unix project * Add comments about configuration mappings * rename solution to powershell * Terminate the loop in PsUtils.GetMainModule() which was infinite in case there was no main module. (PowerShell#6358) * Terminate the loop in GetMainModule if main module is null. * Run tests with FileVersionInfo also on non-windows platforms. Add test for process which main module can be null. * Fix error when 'Format-Wide -AutoSize | Out-String' is called. (PowerShell#6491) * Add checking if an output width is specified. * change _failedToReadConsoleWidth variable to _noConsole bool value. This variable is to cache the default console width when failed to get 'Console.WindowWidth' value. * Add ported Test-Connection cmdlet (PowerShell#5328) The Test-Connection cmdlet works on Windows and Unix. Implemented: Ping Continues Ping Traceroute Detect MTU size (seems don't work on Unix because of .Net Core issue) Connect to TCP port Not jet implemented: I'm putting this off for the future. Detect blackhole routers PingPath Additional considerations As you can see in the tests .Net Core has issues in API implementation. As a result, some tests are skipped on Unix. There is also one issue on Windows. Related comments added to tests. I'm going to open an issue(s) in CoreFX repo. I think we should break the feedback into two parts: scripting functionality interactive functionality (display output) Now I have implemented the output to the screen as the progress bar and as text (without ETS) - we have to decide what is best to use. * Fix running 'pwsh' produced from 'dotnet build' (PowerShell#6549) * Fix line ending in 'DefaultCommands.Tests.ps1' from CRLF to LF (PowerShell#6553) * Engine: Fix several code cleanup issues (PowerShell#6552) * Remove the FullCLR-only symbol-info related code from 'EventManager.cs' (PowerShell#6563) This is because the `AssemblyBuilder` family types do not support emitting symbol information in .NET Core. * Use C# latest language in proj files (PowerShell#6559) Address PowerShell#6547 We begin using C# 7.2 features (Span) but .Net Core doesn't seem use "Latest" as default for a language. So we explicitly set the value. * Use -Throw and -ErrorId native Pester parameters. (PowerShell#6574) * Support 'user@host:port' syntax for SSH Transport (PowerShell#6558) * Update installpsh-<distrofamily>.sh Installers to Handle "preview" in version number (PowerShell#6573) * Improve PSMethod-to-Delegate conversion (PowerShell#6570) Two small improvements: 1. Avoid unnecessary reflection in `ConvertPSMethodInfoToDelegate`. `PSMethod` already has the `MethodInfo` information with its `adapterData` field. 2. Avoid creating the generic `PSMethod<>` type for `PSMethod` that represents constructors, because constructors cannot be converted to a delegate anyways. In case that the `PSMethod` represents constructors, we use a simple `PSMethod` instance instead. Also, rename the type `Unit` to `VOID` to make it more readable, since that type represents `typeof(void)`. * Added check for existence of Location HTTP header before using it (PowerShell#6560) The HTTP RFC (https://tools.ietf.org/html/rfc7231#section-6.4) does not require a Location header to be present for redirects, thus it is required to check if the Location header is returned before using it. * Fix Copy.Item.Tests.ps1 (PowerShell#6596) Fix formatting, absent parameter names and use -Throw and -ErrorID parameters with Should function. * Fix formatting in Convert-Path.Tests.ps1. (PowerShell#6595) * Fix formatting in Clear-Item.Tests.ps1. (PowerShell#6593) * Fix formatting Clear-EventLog.Tests.ps1 (PowerShell#6594) * Fix typos and formatting in Clear-Content.Tests.ps1 (PowerShell#6592) * Fix typos and formatting. * Capitalize function parameters. * Added line break to Acess Denied error message. (PowerShell#6607) * Some fixes in Get-Date -UFormat (PowerShell#6542) * Use UTC datetime in Get-Date -UFormat %s Fix %l output from 0..11 to 1..12 Fix %V using Gregorian calendar * Use a workaround for ISO 8601 week of year (uformat %V) * Add Missing Start-WebListener to Web Cmdlet Tests (PowerShell#6604) * make gem use sudo for macOS (PowerShell#6610) gem install requires sudo on official macOS build VMs. - make bootstrap use sudo for macOS * Create the default PSSession configuration, not tied to a specific PowerShell version. (PowerShell#6519) Create the default PSSession configuration, not tied to a specific PowerShell version. When Enable-PSRemoting command is run, it creates 2 sessions configurations: first, the same as it was before with the name containing the current version expressed as: 'PowerShell.$PSVersionTable.GitCommitId' second with the default name 'PowerShell.6' so that administrators wouldn't have to guess which specific version is installed on the target. PR addresses the issue: PowerShell#6470 * Improve performance of parsing RegexOption for '-split' by using if branches (PowerShell#6605) * Engine: Make some minor cleanup changes (PowerShell#6609) - Remove the calls to `GetTypeInfo()` in the generated `.resx` binding C# binding code (change in the tool `ResGen`). - Remove the unused methods in `SessionState.cs`. - Remove the calls to `GetTypeInfo()` from `Compiler.cs` and other files in `engine\runtime`. - Fix typos in `VariableAnalysis.cs` and `ClassOps.cs`. - Minor perf improvement in `MutableTuple.cs` by using indexing instead of linq extension method `last()`. * Engine: Clean up unneeded 'GetTypeInfo()' calls in interpreter code (PowerShell#6613) Clean up unneeded `GetTypeInfo()` calls in interpreter code. The `GetTypeInfo()` calls were added back in the early days of .NET Core, when most of the properties and methods of `System.Type` were removed. Now those properties and methods are back in `System.Type`, so there is no need to keep those calls. * Make the 'PSISERemoteSessionOpenFile' a support event (PowerShell#6582) Make the `PSISERemoteSessionOpenFile` a support event, so that `Get-EventSubscriber` won't show that subscriber. `Get-EventSubscriber -Force` can still show the support event subscribers. `Unregister-Event -Force` needs to be used to remove a support event subscriber. The event subscriber for `PSInternalRemoteDebuggerStopEvent` and `PSInternalRemoteDebuggerBreakpointUpdatedEvent` are already made support events. And this PR makes it the same for `PSISERemoteSessionOpenFile`. * Fix formatting in Add-Content.Tests.ps1 file. (PowerShell#6591) * Fix formatting. * Capitalize TestDrive and add empty strings. * fix error about setting readonly variable (PowerShell#6617) * Re-order UFormat options in Get-Date (PowerShell#6627) Place in alphabetical order the options * Support importing module paths that end in trailing directory separator (PowerShell#6602) * Remove support for file to opt-out of telemetry, only support env var (PowerShell#6601) Since a PR added support to opt out of telemetry via an environment variable, we can remove the, always intended to be a temporary, solution of deleting a file to opt out of telemetry since the environment variable can be defined at the system level and exist before even installing PowerShell Core. Because the variable is defined as opt out, a value of true, yes, or 1 means no telemetry is sent. * Fix for the Register-PSSessionConfiguration command (PowerShell#6630) Fix for the Register-PSSessionConfiguration command, as some tests were failing in: https://ci.appveyor.com/project/PowerShell/powershell-f975h Probably because of changes in PowerShell#6519 * Clean up 'GetTypeInfo()' calls under engine/parser (PowerShell#6636) `GetTypeInfo()` were added when porting PowerShell to early version of .NET Core (prior .NET Core 1.0). Most properties and methods in `System.Type` were missing at that time. Now the call is not needed anymore and should be removed. * Add meta properties to mac VSTS yml (PowerShell#6619) Add meta properties to mac VSTS YAML - Add a property to clean the build machine - Add a property to set the format of the build name - add a property to set which queue to run the build in * Clean up 'GetTypeInfo()' calls in 'help', 'cimSupport' and 'DscSupport' folders (PowerShell#6633) 'GetTypeInfo()' were added when porting PowerShell to early version of .NET Core (prior .NET Core 1.0). Most properties and methods in 'System.Type' were missing at that time. Now the call is not needed anymore and should be removed. * Clean up 'GetTypeInfo()' calls in other engine sub-folders (PowerShell#6635) 'GetTypeInfo()' were added when porting PowerShell to early version of .NET Core (prior .NET Core 1.0). Most properties and methods in 'System.Type' were missing at that time. Now the call is not needed anymore and should be removed. * Clean up 'GetTypeInfo()' calls in engine folder (PowerShell#6634) 'GetTypeInfo()' were added when porting PowerShell to early version of .NET Core (prior .NET Core 1.0). Most properties and methods in 'System.Type' were missing at that time. Now the call is not needed anymore and should be removed. * Remove extraneous SSH and install docs from the 'demos' folder (PowerShell#6628) * Formatting: Use cache for dash padding strings (PowerShell#6625) * Reformat Format-Table tests (PowerShell#6657) * Convert identations to spaces * Remove alias tests and extra lines * Remane extra Describe * Remove extra parentheses and add spaces * Port Windows PowerShell AppLocker and DeviceGuard UMCI application white listing support (PowerShell#6133) These changes port Windows PowerShell support for Applocker and DeviceGuard User Mode Code Integrity (UMCI) to PSCore6. Windows PowerShell uses public APIs to determine if a system is in locked down mode via AppLocker or DeviceGuard, and automatically runs in constrained language mode. For more information about PowerShell constrained language, see: https://blogs.msdn.microsoft.com/powershell/2017/11/02/powershell-constrained-language-mode/ This support for application whitelisting has mostly existed in PSCore6, but the primary APIs were stubbed out in CorePSStub.cs because they relied on Windows only DeviceGuard (wldp.dll) and AppLocker (Safer APIs) public APIs. These changes re-implement PowerShell lock down APIs on PSCore6 for Windows platforms only. The AppLocker and DeviceGuard public APIs are currently only implemented in Windows OSes and are not supported on Linux or MacOS platforms. Tests have also been ported to PSCore6 and run only for Windows platforms. * Reduce allocations in TableWriter (PowerShell#6648) * [Feature] Reduce allocations in TableWriter * Revert iterator ReadOnlySpan is not supported in iterators * Use columnsThresHold with stackalloc * Add new reliable tests for Get-Date -UFormat (PowerShell#6614) * Add tests for Format-Table -Wrap (PowerShell#6670) * Correct a typo in comment for 'Invoke-WebRequest' (PowerShell#6700) * Add '-WorkingDirectory' parameter to pwsh (PowerShell#6612) Add `-WorkingDirectory` parameter to `pwsh` to allow starting in right working directory. * Pretty print Export-FormatData XML output (PowerShell#6691) Pretty print Export-FormatData XML output by default. Refactor Export-FormatData tests, remove test duplications.

.NET Core has a bug where resolution of "localhost" is very slow (https://github.com/dotnet/corefx/issues/24104). Updated all URL references localhost to 127.0.0.1.

karelz · 2019-10-10T18:27:52Z

Triage: We believe it will be addressed by dotnet/corefx#29716.

karelz · 2019-10-10T18:28:00Z

Duplicate of dotnet/corefx#29716

karelz changed the title ~~HttpWebRequest performance vastly slower in .NET Core 2.0 than full .NET Framework~~ HttpWebRequest hits 1s timeout when resolving http://localhost on Win10 (.NET Framework is fast) Dec 29, 2017

david-pershouse referenced this issue in MammothAU/canopy Feb 13, 2018

Patch remotedriver to use 127.0.0.1 instead of localhost to work around

e124f6f

https://github.com/dotnet/corefx/issues/24104

karelz closed this as completed Oct 10, 2019

msftgits transferred this issue from dotnet/corefx Jan 31, 2020

msftgits added this to the 5.0 milestone Jan 31, 2020

This was referenced Aug 24, 2020

Using 'localhost' makes tests run 10x slower microsoft/WinAppDriver#1269

Closed

Driver is really slow when used with .netcore instead of .netframework appium/dotnet-client#414

Closed

ghost locked as resolved and limited conversation to collaborators Dec 20, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HttpWebRequest hits 1s timeout when resolving http://localhost on Win10 (.NET Framework is fast) #23581

HttpWebRequest hits 1s timeout when resolving http://localhost on Win10 (.NET Framework is fast) #23581

jimevans commented Sep 17, 2017

jimevans commented Sep 17, 2017

davidsh commented Sep 17, 2017

jimevans commented Sep 17, 2017 •

edited

qmfrederik commented Sep 17, 2017 •

edited

Drawaes commented Sep 17, 2017

jimevans commented Sep 18, 2017

Drawaes commented Sep 18, 2017

Drawaes commented Sep 18, 2017

jimevans commented Sep 18, 2017

Drawaes commented Sep 18, 2017

qmfrederik commented Sep 18, 2017

jimevans commented Sep 18, 2017 •

edited

bofcarbon1 commented Oct 9, 2017 •

edited

vincywindy commented Oct 9, 2017 •

edited by karelz

pieter-venter commented Dec 9, 2017

karelz commented Dec 29, 2017 •

edited

davidsh commented Dec 30, 2017 •

edited

jimevans commented Dec 30, 2017

noobed7 commented Jan 11, 2018 •

edited

karelz commented Jan 12, 2018 •

edited

karelz commented Jan 17, 2018

iSazonov commented Feb 1, 2018

davidsh commented Apr 17, 2018

stephentoub commented Apr 18, 2018

geoffkizer commented Apr 18, 2018

karelz commented Oct 10, 2019

karelz commented Oct 10, 2019

HttpWebRequest hits 1s timeout when resolving http://localhost on Win10 (.NET Framework is fast) #23581

HttpWebRequest hits 1s timeout when resolving http://localhost on Win10 (.NET Framework is fast) #23581

Comments

jimevans commented Sep 17, 2017

jimevans commented Sep 17, 2017

davidsh commented Sep 17, 2017

jimevans commented Sep 17, 2017 • edited

qmfrederik commented Sep 17, 2017 • edited

Drawaes commented Sep 17, 2017

jimevans commented Sep 18, 2017

Drawaes commented Sep 18, 2017

Drawaes commented Sep 18, 2017

jimevans commented Sep 18, 2017

Drawaes commented Sep 18, 2017

qmfrederik commented Sep 18, 2017

jimevans commented Sep 18, 2017 • edited

bofcarbon1 commented Oct 9, 2017 • edited

vincywindy commented Oct 9, 2017 • edited by karelz

pieter-venter commented Dec 9, 2017

karelz commented Dec 29, 2017 • edited

davidsh commented Dec 30, 2017 • edited

jimevans commented Dec 30, 2017

noobed7 commented Jan 11, 2018 • edited

karelz commented Jan 12, 2018 • edited

karelz commented Jan 17, 2018

iSazonov commented Feb 1, 2018

davidsh commented Apr 17, 2018

stephentoub commented Apr 18, 2018

geoffkizer commented Apr 18, 2018

karelz commented Oct 10, 2019

karelz commented Oct 10, 2019

jimevans commented Sep 17, 2017 •

edited

qmfrederik commented Sep 17, 2017 •

edited

jimevans commented Sep 18, 2017 •

edited

bofcarbon1 commented Oct 9, 2017 •

edited

vincywindy commented Oct 9, 2017 •

edited by karelz

karelz commented Dec 29, 2017 •

edited

davidsh commented Dec 30, 2017 •

edited

noobed7 commented Jan 11, 2018 •

edited

karelz commented Jan 12, 2018 •

edited